Multimedia Systems (Springer), 2026 Generalized Few-Shot Semantic Segmentation Contrastive Learning Orthogonal Decoupling Feature Decoupling

Generalized Few-Shot Semantic Segmentation via Contrastive Learning and Orthogonal Decoupling

Lulu Jiang1   Yaozheng Xia1   Shaorong Wang1,2

1 School of Information Science and Technology, School of Artificial Intelligence, Beijing Forestry University, Beijing, China
2 Engineering Research Center for Forestry-Oriented Intelligent Information Processing, National Forestry and Grassland Administration, Beijing, China

Paper

TL;DRPropose CLOD-GFSS that integrates multi-scale contextual contrastive learning and hyperspherical orthogonal decoupling to address base-class feature drift and novel-class generalization in GFSS

Abstract

CLOD-GFSS addresses two key limitations of existing GFSS methods: base-class feature degradation during novel-class fine-tuning, and intra-class variations among limited novel-class samples. The method proposes a hierarchical context-aware architecture (CIAM) for multi-scale contrastive learning, and an orthogonal decoupling module (ODM) that constrains feature space geometry via hyperspherical pseudo-target matching.

Method

Existing GFSS two-stage training paradigms cause novel-class fine-tuning to disrupt base-class representations.

Combine contrastive learning with orthogonal decoupling: (1) Multi-scale anchor representations fuse global semantics with local details, optimized via Pixel-Anchor contrastive loss for intra-class compactness and inter-class separation; (2) Hyperspherical orthogonal decoupling generates uniformly distributed pseudo-targets and uses Hungarian algorithm for dynamic anchor-target matching to suppress feature drift.

CIAM

Contextual Information Awareness Module

A multi-scale context fusion mechanism that progressively integrates global semantic features with local detail features through weighted anchor updates. Pixel-Anchor (PA) contrastive loss enforces intra-class feature aggregation around class-specific anchors and inter-class separation across all encoding layers.

ODM

Orthogonal Decoupling Module

Generates uniformly distributed pseudo-target vectors on a d-dimensional hypersphere via Target Generation Loss, then uses Hungarian algorithm for online optimal matching between evolving category anchors and pseudo-targets. Pseudo-Target Matching Contrastive (PTMC) loss enforces feature space topology preservation.

Two-Phase Training

Dual-Phase Progressive Training Strategy

Phase 1: fully supervised base-class learning with CIAM and ODM for discriminative feature space. Phase 2: novel-class update with frozen feature extractor to minimize base-class interference, while CIAM and ODM enhance novel-class feature discriminability through the new classifier.

Fig. 1 Contrastive learning and orthogonal decoupling-based generalized few-shot semantic segmentation model

Fig. 1 Contrastive learning and orthogonal decoupling-based generalized few-shot semantic segmentation model

The overall framework employs a two-phase training strategy: Phase 1 trains the full model (encoder + CIAM + ODM + classifier) on base classes with CE + PA + OrCo losses. Phase 2 freezes the encoder and fine-tunes CIAM, ODM, and a new classifier for novel classes, minimizing interference with base-class representations.

Results

PASCAL-5i (5-shot)
71.86%
Total mIoU
+1.58% over POP (70.28%), SOTA Best overall mIoU on PASCAL-5i 5-shot
PASCAL-5i (1-shot)
39.22%
Novel mIoU
+3.71% over POP (35.51%), best novel-class Significant improvement in novel-class generalization
COCO-20i (5-shot)
49.37%
Total mIoU
+0.62% over POP (48.75%), SOTA Best overall mIoU on COCO-20i 5-shot
COCO-20i (5-shot)
31.15%
Novel mIoU
+0.55% over POP (29.97%), SOTA Best novel-class mIoU on COCO-20i 5-shot
MethodPASCAL-5i 1s NovelPASCAL-5i 5s NovelCOCO-20i 1s TotalCOCO-20i 5s TotalPASCAL-5i 5s Total
CAPL18.8522.4135.4636.8055.72
PKL26.9034.4037.7138.9061.18
DIaM35.1155.3139.0238.5566.97
POP35.5155.8744.9848.7570.28
BCM41.2455.3642.0145.2967.45
CLOD-GFSS ours39.2256.1246.5249.3771.86

CLOD-GFSS achieves state-of-the-art on both benchmarks. On PASCAL-5i 1-shot, novel-class mIoU improves by +3.71% over POP. Ablation shows CIAM adds +0.74% Total mIoU, ODM further adds +3.82% Novel mIoU. Online matching outperforms random assignment by +1.2% Total mIoU. The method reduces boundary errors by 35.7% and improves small-object recall by 28.4% compared to POP.

Qualitative Results

Fig. 2 Multi-scale contextual perception with contrastive learning

Fig. 2 Multi-scale contextual perception with contrastive learning

Takeaway

Combining multi-scale contrastive learning with orthogonal feature space decoupling effectively addresses the dual challenge of base-class preservation and novel-class generalization in GFSS. The key insight is that constraining feature space geometry via orthogonal pseudo-targets on a hypersphere preserves base-class representations while enhancing novel-class discriminability.

Citation

@article{jiang2026clod,
  title={Generalized few-shot semantic segmentation via contrastive learning and orthogonal decoupling},
  author={Jiang, Lulu and Xia, Yaozheng and Wang, Shaorong},
  journal={Multimedia Systems},
  volume={32},
  number={171},
  publisher={Springer},
  year={2026},
  doi={10.1007/s00530-026-02227-8}
}